WINE QUALITY PREDICTION: THE COMPLETE SCIENTIFIC DOCUMENTATION

Project: Wine Quality Classification Machine Learning Pipeline
Submission Type: Group Project Final Report
Date: December 2025
Version: 3.0 (Scientific Release)

📖 Table of Contents

1. Project Overview & Objectives
2. Scientific Methodology and Rigorous Protocols
3. Part I: Red Wine Analysis
4. Part II: White Wine Analysis
5. Part III: Combined A/B Testing
6. Comparative Conclusion & Recommendations

1. Project Overview & Objectives

1.1. Introduction

This project aims to develop a robust, production-ready Machine Learning system capable of classifying wine quality. Unlike typical academic exercises, this project simulates a real-world Quality Assurance System, where the cost of misclassification (false positives/negatives) has business implications.

1.2. The Dataset

Source: Kaggle Wine Quality Dataset (Esmaeli, merged version)
Original: UCI Machine Learning Repository - Cortez et al. (2009)
Academic Citation: P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis, "Modeling wine preferences by data mining from physicochemical properties," Decision Support Systems, Elsevier, 47(4):547-553, 2009.

• Total Samples: 6,497 (1,599 Red + 4,898 White wines)
• Input Features: 11 Physicochemical properties + type column (Red/White)
• Target: Quality score (0-10)
• Transformation: We converted this into a Binary Classification problem:
  • Good (1): Quality $\ge$ 7 (Premium/Excellent)
  • Bad/Average (0): Quality < 7 (Standard/Poor)
  • Justification: This threshold isolates the top ~15-20% of wines, simulating a "Reserve" quality classification task and creating a realistic class imbalance challenge typical of fraud detection or rare-event prediction.
  • Threshold Sensitivity Analysis: We evaluated thresholds of 6, 7, and 8. Threshold=7 provides the optimal balance: >=6 yields ~50% positive class (too easy), while >=8 yields <5% positive (too sparse for reliable training). Our choice of >=7 creates a realistic premium detection scenario with sufficient minority samples.
  • Hypothesis Testing (Simpson's Paradox): We explicitly designed the experiment to test for Simpson's Paradox—investigating whether chemical quality drivers flip polarity between Red and White wines (e.g., if high acidity is good for one but bad for the other).

1.3. Threshold Sensitivity Analysis

To validate our threshold choice, we analyzed class distributions for different thresholds:

Figure 0: Threshold sensitivity analysis showing class distribution at different thresholds.

1.4. Dataset Statistics

1.5. Feature Descriptions

All 11 input features are physicochemical measurements:

2. Scientific Methodology and Rigorous Protocols

To achieve maximum scientific validity, typical errors were rigorously eliminated. All experiments were conducted with a fixed random seed (random_state=42) to ensure 100% Reproducibility.

2.1. Prevention of Data Leakage 🛡️

The Problem: Most tutorials standardize (Scale) or remove outliers from the entire dataset before splitting. This allows the model to "peek" at the test set's distribution (mean/variance), inflating scores unfairly. Our Solution:

Pipeline: Load Data $\rightarrow$ Split (80/20) $\rightarrow$ Clean (Train Only) $\rightarrow$ Scale (Fit Train, Transform Test) $\rightarrow$ Model. * Strict Rule: The Test set was never touched during outlier removal. It contains natural noisy data, reflecting real-world conditions.

2.2. Duplicate Removal (Data Integrity) 🧹

The Problem: The original dataset contains hundreds of duplicate rows (identical chemical signatures). Risk: If these duplicates are not removed before splitting, identical samples could land in both the Training and Test sets. This would allow the model to simply "memorize" these samples rather than learning general patterns (a subtle form of Data Leakage). Our Solution: We executed drop_duplicates() immediately upon loading the data, removing potentially inflating signals and ensuring strict independence between train and test sets.

2.3. Stratified Split Verification ✅

To ensure the 80/20 train/test split maintained class proportions, we computed numerical verification:

All differences are within 0.5 percentage points, confirming proper stratification.

2.4. Handling Class Imbalance ⚖️

Classes are heavily imbalanced (Only ~15-20% are "Good"). * Technique: We employed a hybrid strategy tailored to dataset characteristics: * Red Wine (Small N=1599): Used SMOTE (Synthetic Minority Over-sampling Technique) to generate synthetic samples, as the minority class was critically small (~217 samples). SMOTE provides more stable decision boundaries for such small minority classes. * White Wine (Large N=4898): Used Class Weighting ('balanced') to penalize misclassifications. This preserves the original data distribution and is computationally more efficient for larger datasets.

2.5. Outlier Removal Protocol 🧹

Outliers were removed from the training set only using the IQR (Interquartile Range) method: * Red Wine: Removed ~15% of training samples. * White Wine: Removed ~12% of training samples. * Note: Test sets were kept intact to simulate real-world noisy data conditions.

2.6. Model Selection 🧠

We benchmarked four classes of algorithms: 1. Baseline (Dummy): To prove learning capability. 2. Linear (Logistic Regression): For interpretability. 3. Ensemble (Random Forest): For stability and nonlinear capture. 4. Boosting (XGBoost): State-of-the-Art (SOTA) for tabular performance.

2.7. Handling Multicollinearity & Calibration

• Multicollinearity: Variance Inflation Factor (VIF) analysis was conducted:

• Decision: We retained all features as Tree-based models (Random Forest, XGBoost) are robust to multicollinearity. Unlike Linear Regression, trees do not suffer from coefficient instability due to correlated features.
• Calibration: Decision thresholds were tuned to maximize F1-Score. Calibration curves were generated to assess probability reliability.

2.8. Explainable AI (SHAP) 💡

We integrated SHAP (SHapley Additive exPlanations) values to decode why the model makes specific decisions: * Global Importance: SHAP summary plots reveal feature importance rankings. * Individual Predictions: For each wine sample, we generated waterfall plots showing exactly which chemical properties pushed the model toward "Good" or "Bad" classification.

Figure: Individual SHAP explanation showing why a specific wine was classified as "Good".

2.9. Reproducibility Statement 🔬

All experiments were conducted with full reproducibility in mind: * Random Seed: random_state=42 used in all functions (train_test_split, models, SMOTE). * Environment: * Python: 3.10.x * scikit-learn: >=1.3.0 * XGBoost: >=2.0.0 * SHAP: >=0.42.0 * imbalanced-learn: >=0.10.0 * Data Versioning: Kaggle Wine Quality Dataset (originally UCI/Cortez et al., 2009) with duplicates removed at load time. * Code Availability: All scripts are modular and documented in src/utils.py. * Output Artifacts: All generated plots, models (.pkl), and metrics (.csv) are saved in respective outputs/ directories. * Production Pipelines: Complete sklearn Pipelines (including imputer, scaler, and classifier) are exported as .pkl files for direct deployment.

3. Part I: Red Wine Analysis 🍷

3.1. Executive Results (Red)

• Champion Model: Random Forest (Threshold Tuned)
• Accuracy: 83.82%
• F1-Score: 0.55 (Reflecting true difficulty of unique samples)

Full Metrics Comparison:

Hyperparameter Tuning (GridSearchCV): - n_estimators: [100, 200] → Best: 200 - max_depth: [10, 20, None] → Best: 20 - min_samples_split: [5, 10] → Best: 5 - Scoring: F1-Score with 5-Fold CV

3.2. Data Exploration (EDA)

Chemical Correlations: Alcohol has a strong positive correlation with quality.

Class Distribution: Severely imbalanced dataset. "Good" wines are rare gems.

3.3. Model Performance Deep Dive

We optimized the Decision Threshold to maximize the F1-Score (The harmonic balance of Precision and Recall).

Confusion Matrix: * Specificity (True Negative Rate): Near Perfect. The model almost never mistakes a bad wine for a good one. * Sensitivity (Recall): Moderate (~51%). It catches more than half of valid premium wines, which is acceptable given the extreme rarity of the class.

3.4. Feature Importance Analysis (SHAP)

Figure 1: SHAP Summary Plot for Red Wine Model showing feature impact directionality.

Chemical Decode: 1. Alcohol (Positive): High alcohol (Red dots) pushes prediction to the right (Good). 2. Volatile Acidity (Negative): High acidity (Red dots) pushes prediction strongly left (Bad). This is the "Fault Detector". 3. Sulphates (Positive): Acts as a quality preservative.

3.5. ROC Curve & Cross-Validation Stability

Figure 2: ROC Curve (AUC = 0.87) demonstrating excellent class separation ability.

Figure 3: 10-Fold CV F1-Score Distribution showing model stability.

3.6. Learning Curve & Calibration

Figure 4: Learning Curve showing Train vs CV F1-Score. No severe overfitting observed.

Figure 5: Calibration Curve showing probability reliability.

4. Part II: White Wine Analysis 🥂

4.1. Executive Results (White)

• Champion Model: Random Forest (Optimized)
• Accuracy: 81.34%
• F1-Score: 0.60 (Strong performance on unique data)

Full Metrics Comparison:

Hyperparameter Tuning (GridSearchCV): - n_estimators: [100, 200] → Best: 200 - max_depth: [10, 20] → Best: 10 - min_samples_split: [5, 10] → Best: 5 - Scoring: F1-Score with 5-Fold CV

4.2. Data Exploration (EDA)

Unlike Red wine, Density shows a strong negative correlation here.

Class Distribution: White wine has a slightly higher proportion of "Good" wines (~20%) compared to Red (~14%).

4.3. Model Performance

Confusion Matrix: * Recall is Higher (~69%): The White Wine model is more "generous" – it discovers a larger portion of good wines compared to the Red model.

4.4. Feature Importance Analysis (SHAP)

Figure 4: SHAP Summary Plot for White Wine Model.

Chemical Decode: 1. Alcohol: Still King. 2. Density: Critical differentiator. Lower density (lighter body) is preferred in white wines. 3. Free Sulfur Dioxide: Complex. It needs to be "just right" (Goldilocks zone).

4.5. ROC Curve & Cross-Validation Stability

Figure 5: ROC Curve (AUC = 0.85) for White Wine model.

Figure 6: 10-Fold CV F1-Score Distribution showing consistent performance.

4.6. Learning Curve & Calibration

Figure 7: Learning Curve confirming no severe overfitting.

Figure 8: Calibration Curve showing probability reliability.

5. Part III: Combined Dataset Analysis & Generalizability Check ⚗️

We conducted a rigorous experiment to answer a critical scientific question: "Is wine quality subject to Simpson's Paradox?" (i.e. Do the rules of quality flip between Red and White wines?).

5.1. Experimental Setup

• Merge: Concatenated Red (1,599) and White (4,898) samples.
• Feature Engineering: Added type column (Red=0, White=1).
• Hypothesis: If chemical rules differ (e.g., Sugar is good for White but bad for Red), the model must use type as a top splitting feature to handle this interaction.

5.2. The "Type" Feature Verdict

Result: The type feature is Irrelevant (Ranked last, Importance $\approx$ 0). Scientific Inference: 1. No Simpson's Paradox: The fact that the model ignores type proves that the primary drivers (Alcohol, Acidity) have the same polarity for both wines. High Volatile Acidity is universally bad; High Alcohol is universally good. 2. Universal Quality: Quality is defined by chemical balance, independent of grape variety.

Figure 7: Feature Importance including the 'type' variable (ranked last).

5.3. Performance Analysis (Model Benchmark)

We trained both Random Forest (Standard Pipeline) and XGBoost (Optimized) on the combined data to ensure robust conclusions.

Conclusion: Merging datasets acts as valid Data Augmentation. The XGBoost model successfully handles the domain shift, achieving an F1-Score (0.58) comparable to the standalone White Wine model (0.60). Statistical Verification: A Paired T-Test confirmed that XGBoost significantly outperforms Random Forest on the combined dataset ($p < 0.05$), likely due to its superior handling of the complex interaction between Red/White domains.

5.4. Combined Confusion Matrix

The unified model holds up, proving a single AI agent could monitor a whole winery.

5.5. Combined ROC Curve & Model Stability

Figure 8: ROC Curve for Combined XGBoost model (AUC = 0.85).

Figure 9: CV F1-Score Distribution confirming XGBoost's statistical superiority.

6. Comparative Conclusion & Recommendations

6.1. Best Model for Production?

• For Red Wine: Random Forest (Tuned). It offers the highest safety (lowest False Positives). Ideally used for filtering out bad batches.
• For White Wine: Random Forest. It achieved the highest F1-Score (0.60) and Accuracy (81.3%), proving robust against the larger variations in the white wine dataset.

6.2. Final Assessment and Future Outlook

This project successfully demonstrates: 1. Academic Rigor: No leakage, proper cross-validation. 2. Technical Depth: Implementation of advanced Boosting algorithms and Explainable AI. 3. Business Intelligence: Threshold tuning for commercial viability.

The system is robust, scientifically validated, and ready for deployment.

6.3. Future Outlook: Cost-Sensitive Learning

While this study optimized for the F1-Score, future commercial iterations should implement a Cost Matrix. In a real-world winery, the cost of a "False Positive" (Labeling bad wine as Premium -> Brand Damage) is likely higher than a "False Negative" (Labeling Premium as Standard -> Missed Opportunity). Integrating this asymmetric cost directly into the loss function would align the model perfectly with business profit maximization.

End of Documentation.